Automatic calibration using machine learning

ABSTRACT

There is provided a cell analysis apparatus that comprises image capture circuitry for capturing a brightfield image of a cell using brightfield imaging. The cell has been dyed by a functional dye that indicates, during fluorescence imaging and during brightfield imaging, whether the cell has a given characteristic. A model derived by machine learning is stored and used in combination with the brightfield image to determine whether the cell has the given characteristic. There is also provided a method for creating a cell categorisation model, comprising applying a functional dye to one or more samples comprising a plurality of cells. The functional dye indicates during fluorescence imaging and during brightfield imaging whether each of the cells has a given characteristic. A brightfield image and a corresponding fluorescence image for each of the plurality of cells to which the dye has been applied are captured and a machine learning process is used to generate a model that predicts whether a cell has the given characteristic from a brightfield image. The model is generated by using the brightfield image and the corresponding fluorescence image of each of the plurality of cells as training data.

RELATED APPLICATIONS

This application claims priority to Great Britain Patent ApplicationNumber GB 2108153.4, filed Jun. 8, 2021, the entirety of which is herebyincorporated by reference.

TECHNICAL FIELD

The present disclosure relates to image analysis of biological samplessuch as cells.

BACKGROUND

The number of cells having particular characteristics in a biologicalsample needs to be determined for many procedures in biology andmedicine, such as cell therapy, determining cell viability andin-process controls in industrial bioprocesses. For example in a cellviability process, the total number of cells in a sample along with theratio of alive to dead cells may need to be determined.

Often fluorescence imaging can be used with a functional dye, e.g. a dyethat causes a cell to fluoresce (or not) when the cell has (or lacks) aparticular characteristic or property. However, even though suchfunctional dyes can be very reliable, fluorescence imaging itself can becomplicated and difficult to perform.

SUMMARY

In some examples, there is provided a cell analysis apparatus,comprising: image capture circuitry configured to capture a brightfieldimage of a cell using brightfield imaging, wherein the cell has beendyed by a functional dye that indicates, during fluorescence imaging andduring brightfield imaging, whether the cell has a given characteristic;storage circuitry configured to store a model derived by machinelearning; and processing circuitry configured to use the model incombination with the brightfield image to determine whether the cell hasthe given characteristic.

In some examples, there is provided a method for using a cell analysismodel, comprising: applying a functional dye to a cell to produce a dyedcell, wherein the functional dye is configured to indicate, duringfluorescence imaging and during brightfield imaging, whether the cellhas a given characteristic; capturing a brightfield image of the dyedcell using brightfield imaging; and using a model derived by machinelearning to determine whether the cell has the given characteristic fromthe brightfield image.

In some examples, there is provided a method for creating a cellcategorisation model, comprising applying a functional dye to one ormore samples comprising a plurality of cells, wherein the functional dyeis configured to indicate, during fluorescence imaging and duringbrightfield imaging, whether each of the cells has a givencharacteristic; capturing a brightfield image and a correspondingfluorescence image for each of the plurality of cells to which the dyehas been applied; and using a machine learning process to generate amodel that predicts whether a cell has the given characteristic from abrightfield image, wherein the model is generated by using thebrightfield image and the corresponding fluorescence image of each ofthe plurality of cells as training data.

BRIEF DESCRIPTION OF DRAWINGS

The present technique will be described further, by way of example only,with reference to embodiments thereof as illustrated in the accompanyingdrawings, in which:

FIGS. 1A-1C illustrate a receptacle containing a biological sample inaccordance with some embodiments;

FIG. 2 illustrates an example apparatus in accordance with someembodiments;

FIG. 3 is a flow chart of a calibration method in accordance with someembodiments;

FIG. 4 is a flow chart showing step 310 of the calibration method shownin FIG. 3 in more detail;

FIG. 5 is a flow chart showing step 320 of the calibration method shownin FIG. 3 in more detail;

FIG. 6 is a flow chart of the calibration method shown in FIG. 3 infurther detail;

FIG. 7 is a flow chart showing step 305 of the calibration method shownin FIG. 6 in more detail;

FIG. 8 illustrates an apparatus that is suitable for quickly generatingpairs of fluorescence/brightfield image pairs for the training process;

FIG. 9 shows the production of a spot on an image acquired by thecamera;

FIG. 10 illustrates a flow chart that shows a process of generating thetraining data to be provided to a machine learning algorithm in order togenerate a model;

FIG. 11 shows a similar apparatus to that illustrated in FIG. 8 formaking use of the model that has been devised; and

FIG. 12 illustrates a flow chart that shows a process of applying thegenerated model.

DETAILED DESCRIPTION

Before discussing the embodiments with reference to the accompanyingfigures, the following description of embodiments and associatedadvantages is provided.

In some examples, there is provided a cell analysis apparatus,comprising: image capture circuitry configured to capture a brightfieldimage of a cell using brightfield imaging, wherein the cell has beendyed by a functional dye that indicates, during fluorescence imaging andduring brightfield imaging, whether the cell has a given characteristic;storage circuitry configured to store a model derived by machinelearning; and processing circuitry configured to use the model incombination with the brightfield image to determine whether the cell hasthe given characteristic.

In these examples, the model provides a correspondence between acharacteristic (e.g. a biological characteristic) and brightfield imagesof cells that have been dyed with a functional dye, with the cellshaving the particular characteristic. Such a model can be derived bymachine learning techniques. Fluorescence imaging can be more accuratethan brightfield imaging. However, fluorescence imaging is moredifficult and complicated to perform. The present techniques thereforerelate to the generation of a model (via the use of machine learningtechniques) in which there is a correspondence between a givencharacteristic and a brightfield image. This makes it possible to moreaccurately detect the given characteristic from brightfield imageswithout the need to perform more complicated fluorescence imaging. Thereis no need for the apparatus that uses the model to be the device thattrains the model—although clearly a similarity in equipment (e.g.imaging devices) is likely to lead to more accurate application of thedeveloped model and therefore more accurate predictions using the model.The form of storage need not be non-volatile storage and could includevolatile storage. A functional dye can be considered to be a dye that isused to reveal a particular characteristic of a cell. For instance, afunctional dye might be used to reveal whether a cell is of a particulartype, whether it has a particular configuration, or whether it exhibitsa particular attribute (e.g. being alive or dead, for instance). In somecases such a dye may become visible when the characteristic is presentand in other cases the dye may become visible when the characteristic isnot present. In some cases, the functional dye might begin visible (orinvisible) and the visibility might reverse given a period of time,depending on the presence of the characteristic. Some functional dyesmight be non-binary. For instance, a dye might change to multiplecolours depending on the presence of different characteristics. Theintensity or particular shade of a dye might be used to indicateparticular characteristics—in a similar way to how a pH indicator mightuse different shades to indicate different pHs. In any case, theapplication of the dye is not always entirely unambiguous, although maygenerally be considered to be sufficiently accurate for the purposes ofbiological studies. In these examples, the functional dyes provideindications under both fluorescence imaging and brightfield imaging,although the indication need not be the same in both types of images.

In some examples, the brightfield image is a colour image. As previouslyexplained, some functional dyes might turn different shades depending onthe characteristic being detected in a similar way to how a pH indicatormight turn red in the presence of an acid, and violet in the case of analkali. In these situations, a colour image might be necessary in orderto apply the image to the model in order to determine whether thecharacteristic is present.

In some examples, the brightfield image is a greyscale image. Wherecolour is not essential—for instance where the dye either remains ordoes not remain in a cell, the use of colour might not be of particularuse. In these situations, it might be more useful to instead have a highcontrast image. This can be achieved by using a greyscale brightfieldimage rather than a colour image.

In some examples, the given characteristic of the cell is that the cellis dead. This might, for instance, be determined either by activelydetecting for cell death or by actively detecting for cell liveness.

In some examples, the functional dye is an azo dye. For instance, insome examples, the functional dye is Trypan blue. It is observed thatTrypan blue can be used for the classification of liveness in bothfluorescence imaging and brightfield imaging (albeit it is clearer todistinguish in fluorescence imaging). The Trypan blue dye is expelledfrom live cells after a period of time. In contrast, dead cells areunable to expel the dye. Consequently, after having applied the dye tocells, it is possible to determine whether the cells are alive or deadby whether the cells remain dyed or not. In the case of brightfieldimaging, this can be more difficult to determine since the dye cancontinue to cling to the surface of live cells thereby creating a ‘halo’effect. In contrast, in fluorescence imaging, live cells can bedistinguished from dead cells more easily. Similarly, this can bedifficult to determine since cells transition from alive to dead overtime and so the status of some cells can be ambiguous and difficult fora human to assess consistently.

In some examples, the model has been trained using fluorescence imagesand brightfield images. As previously explained, fluorescence images canbe used to more accurately determine whether a given characteristic ispresent in a cell or not. Consequently, by performing training usingpairs of fluorescence/brightfield images, the brightfield images can beused to perform the training, while the fluorescence imaging is used todetermine whether the given characteristic is present or not. In thisway, a large portion of training data can be automatically generated inorder to train a model to detect the given characteristic frombrightfield images that would be expected to be approximately asaccurate as analysis done using fluorescence imaging, without theassociated complexities of fluorescence imaging being necessary.

In some examples, the model comprises a set of weights or parametersderived by using a neural network. For instance, the neural networkcould be a convolutional neural network. These types of machine learningmodels can be particularly well suited to image analysis.

In some examples, there is provided a method for creating a cellcategorisation model, comprising applying a functional dye to one ormore samples comprising a plurality of cells, wherein the functional dyeis configured to indicate, during fluorescence imaging and duringbrightfield imaging, whether each of the cells has a givencharacteristic; capturing a brightfield image and a correspondingfluorescence image for each of the plurality of cells to which the dyehas been applied; and using a machine learning process to generate amodel that predicts whether a cell has the given characteristic from abrightfield image, wherein the model is generated by using thebrightfield image and the corresponding fluorescence image of each ofthe plurality of cells as training data.

The above method can therefore be used to generate the model thatprovides a correspondence between the characteristic and the brightfieldimages taken of cells to which the functional dye has been applied. Thefluorescence images are used to determine whether the characteristic ispresent or not, obviating the need for human intervention to specifywhether the characteristic is present or not in the correspondingbrightfield image and thereby enabling a large amount of training datato be quickly generated with which a model can be trained. The generatedmodel can then be used in an apparatus (either the same apparatus thatgenerated the model or another apparatus altogether) in order todetermine whether the characteristic exists using brightfield imageswith the dye applied. Since the application of the model (after it hasbeen devised) does not require fluorescence imaging to take place, thecomplexities associated with fluorescence imaging need not take place.Meanwhile the operator of the model can determine whether thecharacteristic is present or not with a higher accuracy andrepeatability than can normally be determined with brightfield imaging.

In some examples, the model comprises a set of weights or parametersderived by using a neural network. For instance, the neural networkcould be a convolutional neural network. These types of machine learningmodels can be particularly well suited to image analysis.

The application could also be configured in accordance with thefollowing paragraphs:

According to one aspect there is provided a calibration apparatuscomprising estimation circuitry configured to estimate, based on acalibration factor, an estimated number of cells of a first type in adyed biological sample containing an unknown number of cells. Thecalibration apparatus also comprises determination circuitry configuredto determine the actual number of cells of the first type in the dyedbiological sample. The calibration apparatus also comprises processingcircuitry configured to adjust the calibration factor. The estimationcircuitry is configured with the processing circuitry to estimate theestimated number of the cells of the first type in the dyed biologicalsample one or more times, based on a different value of the calibrationfactor for each of the one or more times, until the estimated number ofthe cells of the first type approaches the actual number of cells of thefirst type.

In these embodiments, the estimation circuitry uses a calibration factorto perform an estimate of the number of cells of the first type. Theaccuracy of the estimate is dependent on the calibration factor and onceset, the estimation circuitry can be assumed accurate. In order toarrive at the correct calibration factor, the determination circuitry isused to determine the actual number of cells of the first type. By usingdifferent calibration factors, the estimation circuitry can perform anumber of estimates until the estimates that it produces for the numberof cells of the first type approaches the actual value provided by thedetermination circuitry. The estimation circuitry can then be consideredto be calibrated. Future estimates of the number of cells of the firsttype can then proceed using the estimation circuitry alone. Thus, futureestimates of the number of cells of the first type can be produced usinga single circuit (for efficiency). Multiple iterations of thecalibration process might only use the determination circuitry once. Thecalibration process may be considered to be complete when the estimatednumber of cells of the first type is, or (in some embodiments) is withina predefined distance of the actual number of cells of the first type.This predefined distance could be to within an accepted degree of errorsuch as 10%. The dye that is used to dye the biological sample can be,for instance, Trypan blue. However, other possibilities will be known tothe skilled person. The dye is typically one that provides an indicationto differentiate between the different cell types and may providedifferentiation under fluorescent and brightfield imaging.

In some embodiments, the calibration apparatus comprises furtherdetermination circuitry configured to determine the total number ofcells in the biological sample, and wherein the estimation circuitry isfurther configured to estimate, based on the calibration factor, anestimated number of cells of a second type in the dyed biological samplecontaining the unknown number of cells. By determining the total numberof cells and estimating or knowing the number of cells of the first typeit is possible to know (or estimate) the number of cells of a second,complementary type, by subtraction. In some situations, the number ofcells of the first type is calculated for the specific purpose ofdetermining the number of cells of the second type (using the totalnumber of cells as a guide).

In some embodiments the estimation circuitry is further configured withthe processing circuitry to determine the estimated number of cells ofthe first type and the estimated number of cells of the second type inthe dyed biological sample one or more times, based on a different valueof the calibration factor for each of the one or more times, until thesum of: (i) the estimated number of the cells of the first type and (ii)the estimated number of the cells of the second type approaches thetotal number of the cells in the biological sample. This provides afurther mechanism for determining a correct value of the calibrationfactor. In particular, since the total number of cells is known then ifall cells in the sample are of the first type or the second type, thenthis provides a further check that the correct calibration factor hasbeen selected. A substance such as Acridine Orange can be used to stainall cells in a sample, and this makes it possible for all of the cellsto be made visible under fluorescence imaging. Of course, anothertechnique for making all of the cells visible would be to use a pair ofdyes with complementary functions to cause all cells to be made visible.For instance, one could use a first dye that causes live cells to bevisible, and a second dye that causes dead cells to be visible. Such anapproach has the advantage that, provided the two dyes have differentabsorption and/or emission spectra, it is possible to make either typeof cell visible.

In some embodiments the estimation circuitry comprises an opticalmicroscope and an image capturing device configured to capture abrightfield image of the dyed biological sample. This results in asimple and low cost estimation circuitry. Furthermore, using brightfieldimaging (rather than, e.g. fluorescence imaging) can be lesscomplicated, since it does not require the user to perform extrapreparations, ensure that dyes are taken up correctly by cells, orensure that the imaging is performed correctly to capture thefluorescence.

In some embodiments the estimation circuitry is further configured todetermine the estimated number of the cells of the first type and theestimated number of the cells of the second type in the biologicalsample by counting the cells in the brightfield image of the dyedbiological sample. As previously explained, brightfield imaging can beeasier to perform than other forms of imaging. Hence, once calibrationhas been performed, estimations as to the number of cells of the firsttype or the second type can be made with a small amount of complexity.

In some embodiments the estimation circuitry is configured to determinethe estimated number of the cells of the second type by counting thecolourless cells in the brightfield image of the dyed biological sampleand to determine the estimated number of the cells of the first type bycounting the contrasted cells in the brightfield image of the dyedbiological sample. This allows an estimate of the number of cells ofeach type to be easily determined, again using brightfield imaging.

In some embodiments the brightfield image of the dyed biological sampleis a colour image. This can allow for better distinction between cellsand other matter in the biological sample, since more and differenttypes of detail can be illustrated in a colour image.

In some embodiments the brightfield image of the dyed biological sampleis a greyscale image. The use of a greyscale image can create a bettercontrast between cells of the first type and cells of the second type,thereby making the cells of each type easier to distinguish from oneanother. Furthermore, in some cases, without the need to have imagingcapabilities for each colour (red, green, and blue), it may be possibleto achieve a higher pixel density, which can in turn lead to sharperimages that can make it possible to more easily distinguish cell types.

In some embodiments the determination circuitry comprises a fluorescencemicroscope and an image capturing device configured to capture afluorescence image of the dyed biological sample. In some situations,fluorescence imaging can provide a more accurate analysis or count ofcells of particular types. For instance, in the case of Trypan blue,fluorescence imaging causes dead cells to fluoresce while live cells(which do not retain the dye) will not fluorescence. Fluorescing cellsare easier to identify in images and this technique is less prone toerror in terms of identifying the aliveness of cells than brightfieldimaging.

In some embodiments the determination circuitry is configured todetermine the actual number of the cells of the first type in thebiological sample by counting the cells present in the fluorescenceimage of the dyed biological sample. This allows the number of cells ofthe first type to be easily determined.

In some embodiments the determination circuitry is configured todetermine the actual number of the cells of the first type in thebiological sample based on a fluorescent intensity. This allows athreshold to be set for the determination circuitry for what in thefluorescence image corresponds to a cell of the first type. There are anumber of ways of expressing fluorescent intensity. For instance, thiscould be measured as a number of photons within a square area or, in thecase of a digital image, a particular intensity of light (e.g. as agreyscale value) that is received for that cell.

In some embodiments the further determination circuitry comprises anoptical microscope and an image capturing device configured to capture afluorescence image of the biological sample. As previously described, adye such as Acridine Orange (which is visible by fluorescence imaging)can be used to make all cells visible in an image.

In some embodiments the further determination circuitry is configured todetermine the total number of the cells by counting the cells in thefluorescence image of the biological sample. This allows the totalnumber of cells in the biological sample to be easily determined.

In some embodiments the estimation circuitry and determination circuitryare configured to operate simultaneously. By performing such analysis inparallel, the overall time required in order to perform calibration canbe improved over a system in which the estimation circuitry anddetermination circuitry operate one after another.

In some embodiments the second type of cells is living cells. Aspreviously described, living cells can be detected by their ability toexpel Trypan blue after a period of time has elapsed after exposure.Such cells will therefore appear undyed or will retain only a very smallamount of dye around the membrane of the cell (i.e. thereby providing a‘halo’ effect with the dye). Other dyes for identifying the liveness ofcells also exist and may be used in other embodiments. Furthermore,other embodiments may consider other types for the cell characteristicbeyond ‘dead’/‘alive’. For instance, some embodiments might identifycells based on their purpose or function.

In some embodiments the first type of cells is dead cells. In the caseof Trypan blue, a dead cell is not able to expel the Trypan blue fromthe cell and thus, remain solid or substantially solid. Other dyes foridentifying the liveness of cells also exist and may be used in otherembodiments. Furthermore, other embodiments may consider other types forthe cell characteristic beyond ‘dead’/‘alive’. For instance, someembodiments might identify cells based on their purpose or function.

Particular embodiments will now be described with reference to thefigures.

Note that throughout this description, the term “cell” is used. However,the present technique relates equally to particles other than cells,which could be provided in a growth medium or other suspension.

FIGS. 1A, 1B, and 1C illustrate a receptacle 100 containing a biologicalsample 110 in accordance with some embodiments. There are two types ofcell within the biological sample 110, a second type 120A and a firsttype 120B (the two types being mutually exclusive in this case), and thetotal number of cells within the biological sample 110 along with thenumber of cells of the second type 120A and the number of cells of thefirst type 120B is initially unknown. In some examples, the second type120A of cells is living cells and the first type 120B of cells is deadcells. In this example, a dye (e.g. an azo dye such as Trypan blue) isapplied to the biological sample 110 in order to distinguish between thecells of the second type 120A and the cells of the first type 120B. Inthis example, both types 120A, 120B of cell absorb the dye. However,live cells 120A expel the dye while the dead cells 120B do not.Consequently, the dead cells 120B appear contrasted to the cells of thesecond type 120A when viewed under an optical microscope or usingregular (e.g. brightfield) imaging.

An estimation process can be performed, which involves capturing abrightfield image of the dyed biological sample 110, for example usingan optical microscope and an image capturing device, such as a digitalcamera, to capture an image of the biological sample 110 illuminated bywhite light. The brightfield image may be a colour image, where thesecond type 120A of cells appear colourless (e.g. white) and the firsttype 120B of cells appear contrasted, (e.g. coloured by the colour ofthe dye). Alternatively, the brightfield image may be a greyscale image,where the second type 120A of cells appear colourless (e.g. white orlight grey) and the first type 120B of cells appear contrasted (e.g.dark grey or black). The estimated number of cells of the second type120A (e.g. living cells) can be estimated by counting the number ofcells that are uncoloured in the brightfield image and the estimatednumber of cells of the first type 120B (e.g. dead cells) can beestimated by counting the number of cells that are coloured orcontrasted in the brightfield image. This can be done by anyconventional image analysis and processing methodology such as by edgedetection or by looking for groups of contrasted or differently colouredpixels as compared to their surroundings. However, the question as towhether a cell is coloured or not is dependent on the calibration factorand specifically, what counts as ‘coloured’ or ‘contrasted’.

FIG. 1A illustrates an image of the receptacle 100 containing abiological sample 110 as captured during the estimation process. Anumber of cells 120 are present in the biological sample 110, and theyappear either light or contrasted depending on whether they are cells ofthe second type 120A or the first type 120B, and therefore whether theyhave expelled the dye applied to the biological sample. In order todetermine whether the boundary between whether a given cell isdetermined to be a cell of the second type 120A or a cell of the firsttype 120B, a calibration factor is set to determine the shading or levelof brightness at which a cell is determined to be a cell of a particulartype. In the example illustrated in FIG. 1A, cell 120C may be determinedto be a cell of the second type 120A or a cell of the first type 120Bdepending on the calibration factor. The calibration factor can beassociated with the image analysis and processing methodology used withthe brightfield image, for example related to the pixel intensity valueor colour contrast of the brightfield image. For example, thecalibration factor could be proportional to the number and/or wavelengthof photons emitted from the portion of the sample that the image pixelcorresponds to.

In these examples, a determination process involves capturing afluorescence image of the dyed biological sample 110, for example usingan optical microscope and an image camera device, such as a digitalcamera, to capture an image of the biological sample 110 illuminated byfluorescence. The image capture device could be the same image capturedevice as used in the estimation processor or it may be a differentimage capture device. Having performed the fluorescence imaging, thenumber of cells of the first type 120B can be determined as a matter offact by counting the cells present in the fluorescence image of the dyedbiological sample 110. The present technique recognises that Trypan blueis fluorescent and provides a cell characteristic indication under bothfluorescent imaging and brightfield imaging. Consequently, the cells ofthe first type 120B, which retain the dye, are clearly visible on thefluorescence image whilst the cells of the second type 120A, which donot absorb (or which expel) the dye, do not appear (or appear veryfaintly) on the fluorescence image. Accordingly, the cells clearlyvisible on the fluorescence image are the cells of the first type 120B,and therefore by counting all of the cells clearly present in thefluorescence image, the number of cells of the first type 120B in thebiological sample 110 can be accurately determined without the use ofthe previously described calibration factor. The determination of thenumber of cells of the first type present in the fluorescence image ofthe dyed biological sample 110 may be based on a fluorescent intensity.For example, a measure such as photons per area per second is linearlyrelated to the fluorescent intensity. This objective measure sets a cutoff between the range of fluorescent intensities which corresponds to acell of the first type and should therefore be included in the count ofcells of the first type, and the range of fluorescent intensities whichdo not correspond to a cell of the second type and should thus beignored in the count of cells of the second type. However, it will beappreciated that this objective measure typically does not vary undercontrolled conditions, such as when the quantity of cell dye introducedis at a specific concentration, the light use to illuminate the sampleis at a calibrated intensity etc. In particular, the use of fluorescenceimaging to determine cell liveness' provides a more objective measure ofwhether a cell is alive or not than brightfield imaging.

FIG. 1B illustrates an image of the receptacle 100 containing abiological sample 110 as captured by the determination process. A numberof cells 120 are present in the biological sample 110, but as thedetermination process uses fluorescence to capture the image, only thosecells that have retained the dye applied to the biological sample 110will appear clearly in the image, since it is the dye which gives thecells the fluorescent properties. Accordingly, only the cells of thefirst type 120B present in the biological sample 110 can be determinedfrom the determination process. As illustrated in FIG. 1B, cell 120C hasbeen captured in the image generated by the determination process, andtherefore it can be determined that cell 120C is a cell of the firsttype 120B.

The number of cells of the first type 120B in the biological sample 110determined by the determination process can then be compared to theestimated number of cells of the first type 120B in the dyed biologicalsample 110 determined by the estimation process. Since the fluorescenceand brightfield images also correspond (e.g. cover the same view of thesample), it is also possible to perform as a pixel-by-pixel comparisonbetween the two images to see which pixels in the fluorescence imagethat are identified as part of a ‘dead cell’ are similarly identified inthe brightfield image.

The information provided by the determination process can therefore beused to change the calibration factor (possibly repeatedly) used in theestimation process, until the estimation process and the determinationprocess both produce results for the number of cells of the first type120B that approach one another. The term “approaches” is understood tomean that the estimated number of the cells of the first type 120Bmatches the actual number of cells of the first type 120B to within anappropriate margin of error, for example ±5 cells, ±10 cells, ±100cells, ±1000 cells.

For example, consider a situation in which the calibration factorrepresents an intensity of a set of pixels required to consider a cellas ‘dead’. If the estimate falls far below the determination then thecalibration could be adjusted by −10 (making it easier to identify a setof pixels as a dead cell) and a portion of the estimation process couldthen be repeated using the revised calibration factor. If the estimatestill falls sufficiently far below the determination then thecalibration amount might be adjusted by the same amount again.Alternatively, if the new estimated number of the cells of the firsttype 120B is now greater than the actual number of cells of the firsttype 120B and outside the appropriate margin of error, the calibrationfactor may be adjusted to a negative fraction of the amount of theprevious adjustment, such as a half (e.g. +5). In general, any form ofiterative method of adjusting the calibration factor can be employedsuch that the estimated number of the cells of the first type 120Bapproaches the actual number of cells of the first type 120B.

The same brightfield image could be used each time and the calibrationfactor adjusted each time until the estimated number of the cells of thefirst type 120B approaches the actual number of cells of the first type120B. In such an example, the image capturing process for thedetermination process need only be performed once. Alternatively, thecalibration factor may be associated with the optical microscope or theimage capturing device used to generate the brightfield image. In thiscase, an image will be captured each time the process is repeated, basedon a different value of the calibration factor each time the estimationprocess is repeated, and the image analysis and processing methodologyperformed in the same way each time the process is repeated.Accordingly, different portions of the estimation process may berepeated depending on component the calibration factor is associatedwith.

In the example illustrated in FIG. 1B, there are two cells of the firsttype 120B present in the biological sample as it can be determined thatcell 120C is a cell of the first type 120B. If cell 120C is considered,by the estimation process, to be a cell of the second type 120A then theestimation process would only estimate that there is one cell of thefirst type present in the biological sample. Since this would differfrom the actual number of cells of the first type 120B determined by thedetermination process, the calibration factor associated with theestimation process would be adjusted. The estimation process is thenrepeated and the results compared to the determination process withouthaving to repeat the determination process. The estimation process canbe repeated as many times as required until the estimated number ofcells of the first type 120B matches the actual number of cells of thefirst type 120B determined by the determination process. This acts as anindicator that the estimation process is calibrated and the estimationprocess can then be performed on different biological samples withouthaving to perform the determination process on each different biologicalsample. This is advantageous since the estimation process can determinethe number of cells of the second type 120A. This is furtheradvantageous when the estimation process uses brightfield imaging andthe determination process uses fluorescence imaging, because brightfieldimaging is easier to perform.

Since the estimation process and the determination process are performedon the dyed biological sample 110, the estimation process and thedetermination process can be performed simultaneously. In the context ofthe present application, simultaneously is understood to mean that aportion of the estimation process by be performed at the same time as aportion of the determination process. For example, the determinationprocess may be begun whilst the estimation process is being performed,thereby overlapping the running of the processes. Alternatively, theestimation process and the determination process could commence atsubstantially the same time or a portion of the estimation process andthe determination process could be performed before the next portion ofthe estimation process and the determination process is performed. Forexample, the brightfield image could be captured as part of theestimation process and the fluorescence image captured as part of thedetermination process before the number of cells are determined in eachprocess. In some examples where the same image capture device is used tocapture the brightfield image as part of the estimation process and thefluorescence image as part of the determination process, both imagescould be captured by the same image capture device before the images areanalysed in order to determine the number of cells in each image

In some examples, the calibration method further comprises performing afurther determination process on the biological sample 110 to determinethe total number of cells in the biological sample 110. This makes itpossible to perform additional calibration tests. FIG. 1C illustrates animage of the receptacle 100 containing a biological sample 110 ascaptured by the further determination process. A number of cells 120 arepresent in the biological sample 110, but only the total number of cells120 present in the biological sample 110 can be determined from thefurther determination process. That is, the further determinationprocess does not directly determine the types of the cells but byknowing the total number of cells and the number of cells of the firsttype 120B, it is possible to infer the actual number of cells of thesecond type 120A. At least a portion of the estimation process isrepeated one or more times, based on a different value of thecalibration factor for each of the one or more times, until the sum ofthe estimated number of the cells of the first type and the estimatednumber of the cells of the second type approaches the total number ofthe cells in the biological sample. The adjustment of the calibrationfactor and the portions of the estimation process to be repeated are thesame as described above.

There are a number of techniques that can be used for revealing allcells in a biological sample. For instance, a general-purpose dye suchas Acridine Orange can be use in fluorescence imaging to cause all cellsto fluoresce. As an alternatively, a combination of dyes could be used,again in fluorescence imaging—with one dye being used (for instance) tocause live cells to fluoresce and another dye being used to cause deadcells to fluoresce.

The further determination process, in this example, includes capturing afluorescence image of the biological sample 110, for example using animage capture device such as a digital camera (possibly in combinationwith an optical microscope) to capture an image of the biological sample110. The optical microscope may be the same optical microscope as usedin the estimation processor it may be a different optical microscope.The image capture device may be the same image capture device as used inthe estimation process and/or the determination process or it may be adifferent image capture device. The total number of cells in thebiological sample 110 can be determined by counting the number of cellsin the image of the biological sample 110.

The total number of cells in the biological sample 110 determined by thefurther determination process can then be compared to the sum of theestimated number of the cells of the second type 120A and the estimatednumber of cells of the first type 120B determined by the estimationprocess. When the total number of cells determined by the furtherdetermination process and the estimation process are different, thecalibration factor associated with the estimation process can beadjusted in order to improve the accuracy of the estimations in theestimation process. In particular, the total number of cells determinedby the further determination process and the estimation process shouldbe the same if the same biological sample 110 is used. A differencecould be caused, for example, by cell debris or other particles presentin the biological sample 110 being included in the estimates in theestimation process, or by the use of the dye itself causing interferencein the imaging of the estimation process. If a difference exists, theestimation process can be repeated with the adjusted calibration factorand the new estimated total number of cells compared to the total numberof cells determined by the further determination process without havingto repeat the further determination process. The estimation process canbe repeated as many times as required (each time with differentcalibration factors) until the estimated total number of cells matchesthe total number of cells. This acts as another indicator that theestimation process is calibrated and the estimation process can then beperformed on different biological samples without having to perform thefurther determination process on each different biological sample. Thisis advantageous since the estimation process can determine the number ofcells of both the second type 120A and the first type 120B.

Since the further determination process determines the total number ofcells in the biological sample 110 and the determination processdetermines the total number of cells of the first type 120B in thebiological sample 110, the total number of cells of the second type 120Acan be determined by subtracting the number of cells of the first type120B determined in the determination process from the total number ofcells determined in the further determination process. The total numberof cells of the second type 120A can then be compared with the estimatednumber of cells of the second type 120A from the estimation process. Ifthere is a difference then the calibration factor associated with theestimation process can again be adjusted. The estimation process canthen be repeated and the new estimated number of cells of the secondtype 120A compared to the total number of cells of the second type 120Awithout having to repeat the further determination process. Theestimation process can be repeated as many times as required, each timeadjusting the calibration factor, until the estimated number of cells ofthe second type 120A matches the total number of cells of the secondtype 120A. This acts as a further indication that the estimation processis calibrated and the estimation process can then be performed ondifferent biological samples without having to perform the determinationprocess or the further determination process on each differentbiological sample. This is advantageous since the estimation process candetermine the number of cells of both the second type 120A and the firsttype 120B in a single process.

FIG. 2 illustrates an example apparatus 200 in accordance with someembodiments. The calibration apparatus 200 comprises an estimationcircuitry 210 configured to determine an estimated number of cells of asecond type 120A in a dyed biological sample 110 containing an unknownnumber of cells. The calibration apparatus 200 also comprises adetermination circuitry 220 configured to determine the actual number ofcells of the first type 120B in the dyed biological sample 110. Thecalibration apparatus 200 further comprises processing circuitry 250configured to adjust a calibration factor 252 associated with theestimation circuitry 210 when the estimated number of cells of the firsttype 120B is different to the actual number of cells of the first type120B in the dyed biological sample 110.

The estimation circuitry 210 may comprise an optical microscope 212and/or an image capturing device 214 for capturing a brightfield imageof the dyed biological sample 110. The estimation circuitry 210 can thenbe used to determine the estimated number of cells of the second type120A and the estimated number of cells of the first type 120B in thebiological sample 110 by counting cells in the brightfield image of thedyed biological sample 110, for example by counting colourless cells inthe brightfield image of the dyed biological sample 110 and by countingcontrasted cells in the brightfield image of the dyed biological sample110 respectively.

Equally, the determination circuitry 220 may comprise a fluorescencemicroscope 222 and an image capturing device 224 for capturing afluorescence image of the dyed biological sample 110. The determinationcircuitry 220 can then be used to determine the actual number of cellsof the first type 120B in the biological sample 110 by counting cellspresent in the fluorescence image of the dyed biological sample 110.

In some examples, the calibration apparatus 200 also comprises furtherdetermination circuitry 230 configured to determine the total number ofcells in the biological sample 110. The processing circuitry 250 is thenfurther configured to adjust the calibration factor 252 associated withthe estimation circuitry 210 when the sum of the estimated number ofcells of the second type 120A and the estimated number of cells of thefirst type 120B is different to the total number of cells in thebiological sample 110.

The further determination circuitry 230 may comprise an opticalmicroscope 232 and an image capturing device 234 for capturing abrightfield image of the biological sample 110. The furtherdetermination circuitry 230 can then be used to determine the totalnumber of cells by counting cells in the biological sample 110.

The estimation circuitry 210 and the determination circuitry 220 couldbe combined into the same device within the calibration apparatus 200 orcould share components with each other or with other elements of theapparatus 200 in any combination. For example, as described above, theestimation circuitry 210 and the determination circuitry 220 could usethe same image capture device. This is advantageous since the cells inthe biological sample 110 could be in suspension, and therefore it isdesirable to minimise movement of the biological sample 110 in order toavoid influencing the accuracy of the imaging. The further determinationcircuitry 230 could also be incorporated into the same device as theestimation circuitry 210 and determination circuitry 220, furtherreducing the amount of movement of the biological sample 110 required.For example, the estimation circuitry 210 and the further determinationcircuitry 230 may use the same optical microscope whilst the furtherdetermination circuitry 230 may use the same image capture device as atleast one of the estimation circuitry 210 and the determinationcircuitry 220. An automated means of applying the dye to the biologicalsample 110, such as an automated pipetting system, can be incorporatedinto the device in order to further reduce the amount of movement of thebiological sample 110 required.

Although the calibration factor 252 is illustrated in FIG. 2 as formingpart of the processing circuitry 250, in some embodiments it may beseparate from the processing circuitry, for example contained within theestimation circuitry 210 or the component within the estimationcircuitry 210 the calibration factor 252 relates to, such as the opticalmicroscope 212 and the image capturing device 214.

FIG. 3 is a flow chart of the calibration method 300 according to thefirst aspect. The method begins at step 310 where an estimation processis performed on a dyed biological sample containing an unknown number ofcells to determine an estimated number of cells of a first type. At step320 a determination process is performed on the dyed biological sampleto determine the actual number of cells of the first type in thebiological sample. Although step 320 is illustrated as being at the sametime as step 310, these steps may be performed simultaneously asdescribed above or separately. At step 330 the estimated number of cellsof the first type determined by the estimation process is compared tothe actual number of cells of the first type in the biological sampledetermined by the determination process. If the results of theestimation process and the determination process are the same, or withinan appropriate margin of error as described above, the method ends. Ifthe results of the estimation process and the determination process aredifferent or outside the appropriate margin of error, the methodcontinues to step 340 where the calibration factor associated with theestimation process is adjusted and the method returns to step 310, whereat least a portion of the estimation process is repeated with thedifferent, adjusted calibration factor. Steps 330, 340 and 310 of themethod are then repeated as many times as required until the results ofthe estimation process and the determination process are the same, orwithin an appropriate margin of error. Although not shown in the flowchart of FIG. 3 , it will be appreciates that the total number ofiterations could be limited so as to inhibit an infinite loop fromoccurring in a case where there is no calibration factor that achievesacceptable results.

FIG. 4 is a flow chart showing a variant of step 310 of the calibrationmethod 300 shown in FIG. 3 . At step 311 a brightfield image of the dyedbiological sample is captured, for example using an optical microscope212 and an image capturing device 214. At step 312 the estimated numberof cells of the second type in the biological sample is determined bycounting the colourless cells in the brightfield image of the dyedbiological sample captured in step 311 using a current calibrationfactor in order to decide whether each cell is coloured/contrasted ornot. At step 313 the estimated number of cells of the first type in thebiological sample is determined by counting the coloured cells in thebrightfield image of the dyed biological sample captured in step 311.This again makes use of the calibration factor to decide whether a cellis coloured/contrasted or not.

FIG. 5 is a flow chart showing step 320 of the calibration method 300shown in FIG. 3 in more detail. At step 321 a fluorescence image of thedyed biological sample is captured, for example using a fluorescencemicroscope 222 and an image capturing device 224. At step 322 the numberof cells of the first type in the biological sample is determined bycounting the cells in the fluorescence image of the dyed biologicalsample captured in step 321.

FIG. 6 is a flow chart that shows alternative embodiments of thecalibration method 300 shown in FIG. 3 . The method begins at step 309where the biological sample is dyed for the cells of the first type tobe determined. The method then continues to step 310 where an estimationprocess is performed on the dyed biological sample to determine anestimated number of cells of the first type and an estimated number of asecond type in the biological sample. Simultaneously, at step 320, adetermination process is performed on the dyed biological sample todetermine the actual number of cells of the first type in the biologicalsample. A second dying then takes place at step 325 to cause all cellsto be revealed, and at step 327, a further determination process isperformed to determine the actual total number of cells.

Although steps 310 and steps 320, 325, and 327 are illustrated as beingperformed simultaneously, they may be performed sequentially, forexample step 310 prior to step 320, or a portion of step 310 may beperformed before a portion of step 320. However, it is generallydesirable for the estimation process 310 to be complete before thesecond dying takes place at step 325 so as to not corrupt the estimationprocess 310. At step 330 a, the results of the estimation process arecompared to the results of the further determination process bycomparing the sum of the estimated number of cells of the first type andthe estimated number of cells of the second type determined from theestimation process with the total number of cells determined from thefurther determination process. If the results are different, or outsidethe appropriate margin of error, the method continues to step 340, wherea calibration factor associated with the estimation process is adjusted,before the method continues back to step 310 where at least a portion ofthe estimation process is performed again with the different, adjustedcalibration factor. If, at step 330 a, the results of the estimationprocess and the further determination process are the same, or within anappropriate margin of error, the method continues to step 330 b. At step330 b the results the estimation process are compared to the results ofthe determination process by comparing the estimated number of cells ofthe first type from the estimation process with the actual number ofcells of the first type from the determination process. If the resultsare different, or outside the appropriate margin of error, the methodcontinues to step 340, where a calibration factor associated with theestimation process is adjusted, before the method continues back to step310 where at least a portion of the estimation process is performedagain. If, at step 330 b, the results of the estimation process and thedetermination process are the same, or within an appropriate margin oferror, the method continues to step 330 c. At step 330 c the results ofthe estimation process are compared to the results of the determinationprocess by comparing the estimated number of cells of the second typefrom the estimation process with the actual number of cells of thesecond type (i.e. by subtracting the result of the further determinationprocess that determines the total number of cells from the result of thedetermination process that determines the number of cells of the firsttype). If the results are different, or outside the appropriate marginof error, the method continues to step 340, where a calibration factorassociated with the estimation process is adjusted, before the methodcontinues back to step 310 where at least a portion of the estimationprocess is performed again. If, at step 330 c, the results are the sameor within an appropriate margin of error, the method ends and theestimation process is considered to be calibrated. The appropriatemargin of error at each of steps 330 a, 330 b, 330 c could be unique orcould be shared among either or both of the other steps. Havingcompleted the calibration process, the estimation process can then beused to process other biological samples without having to perform thedetermination process or the further determination process again andwithout having to adjust a calibration factor associated with theestimation process again. The ordering of steps 330 a, 330 b, and 330 ccould also be varied.

FIG. 7 is a flow chart showing step 320 of the calibration method 300shown in FIG. 6 in more detail. At step 306 an image of the biologicalsample is captured, for example using an optical microscope 232 and animage capturing device 234. At step 307 the total number of cells in thebiological sample is determined by counting the cells in the image ofthe biological sample captured in step 306.

The calibration method as described above may be implemented by one ormore computers. A computer program may be provided for controlling acomputer to perform the calibration method. A computer readable storagemedium may also be provided for storing the computer program. Thecomputer readable storage medium may be non-transitory. A computerprogram product may also be provided for controlling a computer toperform the calibration method.

According to the above description, it is possible to use a calibrationfactor to adjust a “sensitivity” of a parameter used with a brightfieldimage to determine whether a cell count in the brightfield image matchesa cell count in a fluorescence image. When the calibration factor causesthe two counts to match, it can be said that brightfield imagingproduces an accurate estimate without the need for fluorescence imaging.Further imaging can then take place using brightfield imaging. Althoughthe example of Trypan blue has been provided, which the inventors of thepresent technique have discovered can lead to accurate results inbrightfield imaging, these techniques can also be used with, forinstance, other functional dyes. For instance, a functional(fluorescent) dye could be used to characterise the mitotic index ofcells, which can then be determined in brightfield imaging according tothe size of those cells. The calibration factor could be, for instance,a radius of a cell. The fluorescence imaging would produce a ‘truth’ asto the actual mitotic index and the calibration factor could be modifieduntil the mitotic index estimated using brightfield imaging corresponds(within the accepted level of error) to the mitotic index determined bythe fluorescence imaging.

The calibration factor could be determined according to a series ofweights or parameters determined using machine learning. In suchexamples, an apparatus is used to quickly generate images of biologicalsamples that have been stained with a functional dye such as Trypan blueusing both fluorescence imaging and brightfield imaging to generatefluorescence/brightfield image pairs. These pairs of images can be usedin order to produce a training set. In particular, within each pair, thefluorescence image provides the answer as to whether an image actuallycontains any live cells or not (and/or how many live cells are present)while training is carried out using the corresponding brightfield image.Having performed such calibration, the resulting model can be used todetermine, with a high success rate, whether a given sample containslive cells using only a brightfield image of dyed cells. Havingidentified the live cells, it is also possible to count the number oflive cells within the image.

FIG. 8 illustrates an apparatus that is suitable for quickly generatingpairs of fluorescence/brightfield image pairs for the training process.A similar apparatus was originally described in more detail in grantedpatent GB2495537, the contents of which are incorporated herein in theirentirety.

As shown in FIG. 8 , the apparatus 10 includes an XY stage 20 on which amicrowell container 12 can be mounted. The XY stage 20 includes a Xmotor 22 and associated encoder 24 and feedback circuit 26 which can becontrolled by a computer 28 to move the container 12 in the X direction,i.e. to the left and right as seen in FIG. 8 . The XY stage 20 alsoincludes a Y motor 30 and associated encoder 32 and feedback circuit 33which can be controlled by the computer 28 to move the container 12 inthe Y direction, i.e. in and out of the paper as seen in FIG. 8 . Theapparatus furthermore includes an optical system for illuminating aportion of the container and acquiring images of the illuminatedportion.

Specifically, the optical system includes, in order along an opticalaxis 34 above the container 12: a light source 35 which may beimplemented by an LED: a collecting lens 36; a collimating lens 37; anaperture 38 which may be adjustable; a mirror 40; a focusing lens 42; anaperture 44 which may be a fixed aperture or may be one of severalapertures provided in a wheel 46 or strip movable by a motor 48 toselect a particular aperture; and a condenser lens 49 which projects thelight from the source 35 onto the biological sample in the microwellcontainer 12.

The optical system furthermore includes, in order along an optical axis34 below the container: an objective lens 50 which is mounted on amechanical slide 52 including a motor 54 and associated encoder 56 andfeedback circuit 58 which can be controlled by the computer 28 to movethe objective lens 50 in the Z direction, i.e. up and down as seen inFIG. 8 : a beam splitter 59 which may be implemented by a half-silveredmirror: a focusing lens 60; and a digital monochrome or colour camera62.

In addition to being able to illuminate the sample and acquire imagesthereof, the optical system is also arranged to enable a determinationto be made of the height of the portion of the floor 16 of the container12 which is currently in the view of the camera 62 so that the focus ofthe image acquired by the camera 62 can be appropriately set.Specifically, a laser source 64 is disposed behind the beam splitter 59and projects a laser beam 66 through the beam splitter 59 and theobjective lens 50 so that it impinges at an inclined angle on thecontainer 12 adjacent the optical axis 34. A first reflected beam 68 isproduced at the underside of the container 12 and is reflected backthrough the objective lens 50 and passes into the camera 62 where, asshown in FIG. 9 , it produces a first spot 70 in the image 72 acquiredby the camera 62. Also, in the regions of the compartments 14 of thecontainer 12, a second reflected beam 74 is produced at top side of thefloor of the container 12 and is reflected back through the objectivelens 50 and passes into the camera 62 where, as shown in FIG. 9 , itproduces a second spot 76 in the image 72 acquired by the camera 62.

For the purposes of fluorescence imaging, a fluorescence light source 63can also be added to illuminate samples in the compartments 14 of thecontainer 12 with light of a given wavelength. The biological sample may(depending on the characteristics of the imaging being performed)generate light of a difference wavelength. As will be known to theskilled person, when fluorescence imaging occurs, a filter could beemployed (e.g. within the camera 62) to filter out light other than thelight generated by the biological sample.

The captured images can be processed by processing circuitry 65 in orderto generate a model using machine learning to determine whether abrightfield image contains live cells (and/or to count the number oflive cells within a captures image). The generated model can be storedin storage circuitry 67.

This system can be used to quickly perform centralisation of images thatare taken, as well as focusing of the images that are taken. In respectof centralisation, since the compartments 14 are curved, the distancebetween the objective lens 50 and the floor 16 of the compartment willincrease and decrease as the XY stage 20 moves left and right. Thiscauses the reflected spots 70, 76 in the image 72 to move (e.g. alongthe U axis in FIG. 9 ). By moving the XY stage 20 so that the spots 70,76 are approximately in the centre of the image 72, the compartment 14should be centralised within the image. Focusing on the contents of thecompartments 14 can also be achieved quickly. A first initialisationstep is performed by using brightfield imaging to take a series ofimages of a biological sample at different Z positions of the objectivelens 50. A determination is then made (either automatically or from auser) as to which of the images (i.e. which Z position of the objectivelens 50) produces the best focus on the contents of the compartment 14.Automated focusing for other compartments 14 (whose floors 16 arecurved) can then take place. In particular, there is a linearrelationship between the heights (e.g. Vc1, Vc2) along the V axis of thespots 70, 76 produced in the image 72 and the Z position of theobjective lens 50. The relationship can be determined by considering thegradient of the change in V for a given spot as compared to the changein Z for the objective lens 50. It is then possible, for a givencompartment 14 to adjust the Z position of the objective lens 50 tomatch the Z position that was determined as produced the best focus onthe contents of the compartment 14 during the initialization step.

When the above apparatus is used to image cells that have been dyedusing the functional dye Trypan blue, dead cells remain coloured by thedye and live cells expel the dye. Trypan blue is visible underbrightfield imaging using the imaging apparatus 62 (which could eitherbe a coloured imaging apparatus or a greyscale imaging apparatus).However, the dye has increased visibility under fluorescence imagingwhere dead cells are shown as solid masses. Live cells, which have beenable to remove the Trypan blue, are either not shown in such images orare shown as ‘halos’. It is therefore possible to identify live cells bylooking for clusters of pixels. Having identified these regions ofinterest, it is possible to use machine learning to look for identifyingfeatures in the corresponding brightfield image that was takenapproximately simultaneously. A training set of images might containtens or possibly hundreds of pairs of fluorescence/brightfield images. Anumber of machine learning algorithms can be used. However, in someembodiments, a neural network such as a convolutional neural network(such as regions with convolutional neural networks, R-CNN) can be usedto perform the training.

FIG. 10 illustrates a flow chart 500 that shows a process of generatingthe training data to be provided to a machine learning algorithm inorder to generate a model. Prior to this process the microwell container12 is provided in which the samples (e.g. cells) in the wells arestained with a functional dye such as an azo dye (e.g. Trypan blue). Ata step 502, the focal plane is determined using the laser as previouslydescribed. This could take place using human intervention, for instance,by considering the first well of the microwell container 12. Havingdetermined which of several images has the greatest focus, the Zposition of the objective lens 50 is set. At a step 504, the XY stage 20is moved so that the objective lens is centred on the next compartment14 of the microwell container 12. At a step 506, the objective lens isadjusted so that it focusses on the focal plane identified in step 502,taking into account the curved floor 16 on which the sample rests. Forinstance, if the floor 16 is slightly raised as compared to the floor ofthe initial compartment (as determined from the laser) then theobjective lens 50 is moved so that the chosen plane remains as the focalplane. At a step 508, a pair of images (one brightfield image, onefluorescence image) are produced at substantially the same time. At step510, it is determined whether further compartments remain. If so, theprocess returns to step 504. Otherwise, at step 512, each brightfieldimage is classified using the fluorescence image (e.g. whether there arelive cells, where the live cells are, how many live cells are presentetc.). The information, together with the brightfield image is thenprovided to the machine learning algorithm in step 514 to performmachine learning and generate a model that, e.g. indicates whether abrightfield image contains live cells (or indicates the number of livecells, or indicates whether a given brightfield image is of a livecell).

FIG. 11 shows a similar apparatus to that illustrated in FIG. 8 formaking use of the model that has been devised. In this example, thestorage circuitry 67 stores the model that has been devised and theprocessing circuitry 65 uses the model on a captured brightfield image.For instance, if the model determines whether a given brightfield imagecontains a live cell then the model will provide this information whengiven a brightfield image of a cell that has been dyed with Trypan bluein the same manner in which the model was initially produced. Note thatin this embodiment, no fluorescent light source is provided becausefluorescence imaging is no longer required. Thus, the complexity anddifficulty involved with fluorescence imaging can be avoided while stillmaintaining a high accuracy when performing image analysis onbrightfield images.

FIG. 12 illustrates a flow chart 600 that shows a process of applyingthe generated model. Prior to this process, a microwell container 12 isprovided in which the samples (e.g. cells) in the microwell container 12are stained with a functional dye such as an azo dye (e.g. Trypan blue).At a step 602, the focal plane is determined using the laser aspreviously described. This could take place using human intervention,for instance, by considering the first well of the microwell container12. Having determined which of several images has the greatest focus,the Z position of the objective lens 50 is set. At a step 604, the XYstage 20 is moved so that the objective lens is centred on the nextcompartment 14 of the microwell container 12. At a step 606, theobjective lens is adjusted so that it focusses on the focal planeidentified in step 602, taking into account the curved floor 16 on whichthe sample rests. For instance, if the floor 16 is slightly raised ascompared to the floor of the initial compartment (as determined from thelaser) then the objective lens 50 is moved so that the chosen planeremains as the focal plane. At a step 608, a brightfield image isproduced. At step 610, the model is applied to the brightfield image inorder to produce a result. At step 612, it is determined whether furthercompartments 14 remain. If so, the process returns to step 604.Otherwise, the process ends at step 614.

Consequently, it can be seen how machine learning can be applied toforgo the complexities of brightfield imaging by using brightfield,fluorescence image pairs to train a model. The model can then be appliedto brightfield images to gain at least some of the improved accuracythat can be achieved with fluorescence imaging, without the complexsetup required for fluorescence imaging.

Although the above description has focused on the use of Trypan blue, itwill be appreciated that the same process can be carried out using otherazo dyes or indeed other functional dyes that can be used to indicate aparticular characteristic of a cell. Clearly, when the above techniqueis used to identify whether a given cell is alive or not, the sameprocess can be repeated for multiple cells in a brightfield image inorder to determine how many live cells exist.

In the present application, the words “configured to . . . ” are used tomean that an element of an apparatus has a configuration able to carryout the defined operation. In this context, a “configuration” means anarrangement or manner of interconnection of hardware or software. Forexample, the apparatus may have dedicated hardware which provides thedefined operation, or a processor or other processing device may beprogrammed to perform the function. “Configured to” does not imply thatthe apparatus element needs to be changed in any way in order to providethe defined operation.

Although illustrative embodiments of the invention have been describedin detail herein with reference to the accompanying drawings, it is tobe understood that the invention is not limited to those preciseembodiments, and that various changes, additions and modifications canbe effected therein by one skilled in the art without departing from thescope and spirit of the invention as defined by the appended claims. Forexample, various combinations of the features of the dependent claimscould be made with the features of the independent claims withoutdeparting from the scope of the present invention.

What is claimed is:
 1. A cell analysis apparatus, comprising: image capture circuitry configured to capture a brightfield image of a cell using brightfield imaging, wherein the cell has been dyed by a functional dye that indicates, during fluorescence imaging and during brightfield imaging, whether the cell has a given characteristic; storage circuitry configured to store a model derived by machine learning; and processing circuitry configured to use the model in combination with the brightfield image to determine whether the cell has the given characteristic.
 2. The cell analysis apparatus of claim 1, wherein the brightfield image is a colour image.
 3. The cell analysis apparatus of claim 1, wherein the brightfield image is a greyscale image.
 4. The cell analysis apparatus of claim 1, wherein the given characteristic of the cell is that the cell is dead.
 5. The cell analysis apparatus of claim 1, wherein the functional dye is an azo dye.
 6. The cell analysis apparatus of claim 1, wherein the functional dye is Trypan blue.
 7. The cell analysis apparatus of claim 1, wherein the model has been trained using fluorescence images and brightfield images.
 8. The cell analysis apparatus of claim 1, wherein the model comprises a set of weights or parameters derived by using a neural network.
 9. The cell analysis apparatus of claim 8, wherein the neural network is a convolutional neural network.
 10. A method for using a cell analysis model, comprising: applying a functional dye to a cell to produce a dyed cell, wherein the functional dye is configured to indicate, during fluorescence imaging and during brightfield imaging, whether the cell has a given characteristic; capturing a brightfield image of the dyed cell using brightfield imaging; and using a model derived by machine learning to determine whether the cell has the given characteristic from the brightfield image.
 11. A method for creating a cell categorisation model, comprising: applying a functional dye to one or more samples comprising a plurality of cells, wherein the functional dye is configured to indicate, during fluorescence imaging and during brightfield imaging, whether each of the cells has a given characteristic; capturing a brightfield image and a corresponding fluorescence image for each of the plurality of cells to which the dye has been applied; and using a machine learning process to generate a model that predicts whether a cell has the given characteristic from a brightfield image, wherein the model is generated by using the brightfield image and the corresponding fluorescence image of each of the plurality of cells as training data.
 12. The method of claim 11, wherein the machine learning process comprises the use of a neural network to generate the model.
 13. The method of claim 12, wherein the neural network is a convolutional neural network.
 14. A non-transitory storage medium comprising the cell categorisation model produced according to the method of claim
 11. 